Improving Transformer Based End-to-End Code-Switching Speech Recognition Using Language Identification

A Recurrent Neural Networks (RNN) based attention model has been used in code-switching speech recognition (CSSR). However, due to the sequential computation constraint of RNN, there are stronger short-range dependencies and weaker long-range dependencies, which makes it hard immediately switch languages CSSR. Firstly, deal with this problem, we introduce CTC-Transformer, relying entirely on a self-attention mechanism draw global adopting connectionist temporal classification (CTC) as an auxiliary task for better convergence. Secondly, proposed two multi-task learning recipes, where language identification (LID) is learned addition CTC-Transformer automatic (ASR) task. Thirdly, study decoding strategy combine LID into ASR Experiments SEAME corpus demonstrate effects methods, achieving mixed error rate (MER) 30.95%. It obtains up 19.35% relative MER reduction compared baseline RNN-based CTC-Attention system, 8.86% system.